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Introduction 


There is growing interest in providing students and 
others with information about the performance of 
postsecondary education and training programs, and 
rightfully so. Currently, if one wants to know how many 
students get jobs and what those jobs pay, one cannot 
find comparable information for programs across the 
nation. Consumer reports, which display information 
about programs and their outcomes — such as the 
percentage of students who complete their programs 
and find jobs, and average earnings of graduates — can 
help students, policymakers, and others make more 
informed decisions. 


To their credit, the U.S. Department of Education (ED) 
through its College Scorecard and the sponsors of the 
College Transparency Act (CTA) now being considered 
by Congress, are attempting to address this information 
gap by providing online consumer reports. The President 
recently issued an executive order that the College 
Scorecard should report outcomes at the program-level. 
As these initiatives continue and evolve, it is important 
to align consumer reports with the characteristics and 
goals of today’s students and postsecondary programs. 


A large and growing share of postsecondary students 
are working adults who attend certificate and associate 
degree programs that can quickly advance their level of employment and earnings. Working adults 

often do not have time for years of additional education. Sixty-four percent of undergraduate students 
work and 40 percent work full-time. Seventy-four percent of subbaccalaureate credentials are from 
occupational programs designed to prepare students for employment as opposed to academic 
credentials designed for transfer to four-year institutions.* An estimated 38 percent of all undergraduate 
credentials from Title IV higher education institutions? are occupational associate degrees or certificates. 
Moreover, these numbers do not include the large number of non-credit training programs at community 
and technical colleges.* An estimated 41 percent of enrollments at public two-year colleges are in non- 
credit programs.*® 


For consumer reports to meet the needs of today’s students, many of whom enroll in subbaccalaureate 
occupational programs, it’s critical that the reports show labor market outcomes shortly after students 
graduate. If enacted by Congress, CTA could address these issues.® In the meantime, to ensure that 
students and policymakers have comparable information about programs’ immediate outcomes, the 
College Scorecard consumer pages should include a measure of earnings shortly after graduation. 


| “Today’s Student,” Lumina Foundation website, 2019. 

2 Lisa Hudson, “Trends in Subbaccalaureate Occupational Awards: 2003-2015,” National Center for Education Statistics, 2018. 

3 Institutions eligible for Title IV financial aid under the Higher Education Act (HEA), such as Pell grants and loans. 

4 The data reported by the National Center for Education Statistics is from the Integrated Postsecondary Education Data System (IPEDS) which does not 
collect data on non-credit programs. 

“Fast Facts 2018,” American Association for Community Colleges website. 

5 CTA allows the National Center for Education Statistics four years after enactment to create the new consumer report system. 
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Current landscape of 
consumer reports 


Subbaccalaureate occupational education and 
training programs are already required by statute 
to post online consumer reports. ED should 
consider these Congressional requirements as it 
updates the College Scorecard and implements 
CTA, if it is enacted. 


The 2014 Workforce Innovation and Opportunity 
Act (WIOA) requires states to make available by 
electronic and other means easily understandable 
performance reports for eligible providers of 
training services.’ These reports must include 

the performance for each program of study that 
institutions or other providers want to be on 

the state’s Eligible Training Provider (ETP) list 

of programs eligible to receive funding through 
WIOA Title | Individual Training Account vouchers. 
Reported performance must be based on 
outcomes for all students, not just those funded by 
WIOA. 


Similarly, the 2018 Strengthening Career and 
Technical Education for the 21st Century Act 
(commonly referred to as, Perkins V) requires 
institutions receiving Perkins funds to develop 
and disseminate a performance report “through a 
variety of formats, including electronically through 
the Internet, to students, parents, educators, 

and the public.”® These reports must include 

the performance for each Career and Technical 
Education (CTE) program of study? at institutions 
receiving Perkins funds, and must be based on 


outcomes for all CTE concentrators in the program. 


Perkins V becomes effective on July 1, 2019. 


A large share of postsecondary programs is 
covered by these WIOA and Perkins requirements. 
Perkins V covers virtually every public 
postsecondary CTE program in the nation. 
Approximately 80 percent of community colleges 
7 WIOA: Sec.116(d)(4) 


have one or more programs ona state’s WIOA ETP 
list.° Some state two-year systems automatically 
submit all workforce education programs to be 

on their state’s ETP list." The WIOA requirements 
also cover private providers, including non-profit 
institutions and proprietary schools, that want to 
be eligible for their state’s ETP list. 


While not required by statute, ED has created the 
online College Scorecard. The College Scorecard 
provides consumers with basic information about 
Title lV postsecondary institutions, including 

the earnings of students after leaving their 
institution. The information covers institutions, not 
individual programs at institutions. ED, however, 
has indicated they plan to publish program- 

level information on the College Scorecard or a 
successor site,!* and on March 21, the President 
issued an Executive Order directing ED to do so." 


In addition to not reporting information at the 
program-level, another major drawback of ED’s 
College Scorecard is that its post-college outcomes 
data are based solely on students receiving Title 
IV financial aid.% Nationwide, just an estimated 70 
percent of students at Title IV eligible institutions 
receive some such aid,* and this percentage 
varies widely among intuitions. Only 39 percent 
of students at public two-year institutions receive 
Title IV financial aid. In California, only about 20 
percent of all community college students receive 
Title IV aid.*© The College Scorecard is therefore 
not a valid representation of results among all 
students. 


To address this problem, bipartisan members of 
Congress are sponsoring CTA. CTA would direct 
the Commissioner of Education Statistics to create 
a consumer report system including all students 

at Title IV institutions, whether or not they receive 
federal student aid. Based on privacy-protected 
student-level data, the Commissioner would 

create a consumer-friendly website showing key 
information about programs at Title IV institutions. 


8 Strengthening Career and Technical Education for the 21st Century Act, Sec. 113(b)(4)(B)(v) 

° “Except that in a case in which reporting by such program or program of study is impractical, the data may be disaggregated by the career clusters of 
the CTE concentrators.” Strengthening Career and Technical Education for the 21st Century Act, Sec. 113 (4)(B)(ii)(III)) 

© Author's calculation based on estimates from Social Policy Research Associates. 

" For example, Washington’s community and technical colleges apply for all workforce education programs to be listed on the state’s Eligible Training 


Provider List. 


® August 14, 2018 Notice of Proposed Rulemaking, FR Document 2018-17531. 83 FR 40167. 
8 The White House, “Executive Order on Improving Free Inquiry, Transparency, and Accountability at Colleges and Universities,” March 21, 2019. 
“ There are a number of additional drawbacks to the College Scorecard. For instance, the reported earnings outcomes include both students who did not 


graduate and graduates. 


® Using Federal Data to Measure and Improve the Performance of U.S. Institutions of Higher Education,” Executive Office of the President, 2017. 


© California Community Colleges Chancellors Office, Data Mart for 2017-18. 
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Information would include post-program outcomes 
including employment, earnings, and further 
education. 


In addition to these federal efforts, at least nine 
states have their own program-level higher 
education consumer report system that reveals 
post-program labor market outcomes.” While 
these reports are very useful, a drawback to most 
of these state reports is they only include post- 
program employment information on graduates 
employed in their state. In contrast, the College 
Scorecard includes information on employment no 
matter where someone is employed; as would CTA. 


Performance measures 


WIOA and Perkins V establish mandatory 
performance measures for consumer reports in 
statue. For WIOA, the four mandatory performance 
measures for consumer reports are: 


|. The percentage of program participants 
who are in unsubsidized employment 
during the second quarter after exit from 
the program; 


Il. The percentage of program participants 
who are in unsubsidized employment 
during the fourth quarter after exit from the 
program; 


Ill. The median earnings of program partici- 
pants who are in unsubsidized employment 
during the second quarter after exit from 
the program; 


IV. The percentage of program participants 
who obtain a recognized postsecondary 
credential, or a secondary school diploma 
or its recognized equivalent ... during par- 
ticipation in or within 1 year after exit from 
the program.!® 


Y CO, FL, KT, Ml, MN, MO, NC, TX, and WA. 
® WIOA Sec.116(b)(2)(A)(i). 


Interagency teams from the states developed 
these performance measures through a lengthy 
process.’? The measures were designed to be useful 
for both consumer information and performance 
accountability for a wide-variety of workforce 
development and education programs.” Follow-up 
periods were chosen that are indicative of longer- 
term results, but short enough to provide useful 
information about the performance of programs 
and strategies that are likely to still be in place. The 
follow-up periods also reflect that most program 
participants in occupational programs want to 
quickly advance their employment and earnings. 


In 2018, Congress aligned some Perkins V 
measures with WIOA requirements. For Perkins 
V, the mandatory performance measures for 
postsecondary CTE, including institutions’ 
consumer reports are: 


“(i) The percentage of CTE concentrators 
who, during the second quarter after program 
completion, remain enrolled in postsecondary 
education, are in advanced training, military 
service, or a service program that receives 
assistance under title | of the National and 
Community Service Act of 1990 (42 U.S.C. 
12511 et seq.), are volunteers as described 

in section 5(a) of the Peace Corps Act (22 
U.S.C. 2504(a)), or are placed or retained in 
employment. 


(ii) The percentage of CTE concentrators who 
receive a recognized postsecondary credential 
during participation in or within 1 year of 
program completion. 


(iii) The percentage of CTE concentrators in 
career and technical education programs and 
programs of study that lead to non-traditional 
fields.” 


Important to this discussion, Perkins V requires 
institutions’ consumer reports to disaggregate the 
outcome information included in the first measure. 
For example, they must report the percentage 

of CTE concentrators who are employed after 
completion, not just the total percentage who 

are either employed, in the military, in further 


® “Integrated Performance Information for Workforce Development: A Blueprint for the States;” Workforce Training and Education Coordinating Board; 


Olympia, Washington; 2005. 


2° The four WIOA performance measures have been widely incorporated across federally-funded programs. In WIOA, Congress applied the four measures 
to eleven national employment and training programs administered by the U.S. Department of Labor or DOE. Since WIOA, Congress has also applied 
the measures to the Trade Adjustment Assistance Act and Supplemental Nutrition Assistance Program (SNAP) Employment and Training. 

21 Strengthening Career and Technical Education for the 21st Century Act, Sec.113(b)(2)(B). 
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be 


education, or in a national service program. 

They must also report the percentage of CTE 
concentrators who enrolled in further education or 
training after completion.”2 


There are no statutory requirements for the 
performance measures in the College Scorecard; 
they were established by administrative action. 
While the College Scorecard allows access to 
reams of data that skilled users can manipulate to 
produce a variety of information, there is limited 
information on the consumer-facing page that 
viewers first see for each institution. The two 
performance measures on the consumer-facing 
pages of the College Scorecard are: 


® Graduation rate within 150% of the 
expected time of completion 


® Median earnings of former students 10 
years after entering the school 


CTA does not establish specific performance 
measures. The closest the language comes 

to specifying the measures of post-program 
employment and earnings is that they should 
be measured, “immediately after leaving 
postsecondary education; and at later time 
intervals appropriate to the credential sought 
and earned.’”*3 The Commissioner of Education 
Statistics would develop the specific metrics 
through consultation with stakeholders. 


The below table shows the main consumer report 
performance measures for WIOA training providers, 
Perkins V postsecondary CTE, and the College 
Scorecard consumer-facing-page. Additional 
information they require or provide are listed in the 
Appendix. 


22 Strengthening Career and Technical Education for the 21st Century Act, Sec.113(4)(B)(ii)(IV). 


23 College Transparency Act, Sec.2(b)(4)(C)(v)(II). 
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Postsecondary consumer reports 


Completion Rate Continued Education Employment Earnings 


WIOAEligible | Community Colleges, Percentage of Not required (1) Percentage of Median earnings of 
Training Proprietary Schools, participants who participants employed — | program participants 
Providers (ETP) | apprenticeship obtain a recognized during the 2™ quarter during the second 

programs, Community- | postsecondary after exit: quarter after exit 

Based Organizations and | credential during 

other providers eligible | participation in or (2) Percentage of 

to train individuals within 1 year after exit participants employed 

funded by WIOA during the 4" quarter 

after exit 


Perkins Local | CTE programs at Percentage of Percentage of students | Percentage of students | Not Required 
Recipient community colleges or | students who who are enrolled employed during the 2™ 
[Institution] other postsecondary receive a recognized in postsecondary quarter after completion 
Report institutions receiving postsecondary education or training, 
Perkins funds. credential during during the second 
participation in quarter after program 
or within 1 year of completion. 
program completion 


College Undergraduate degree- | Graduation rate within | Not Reported Not Reported Median earnings of 
Scorecard granting institutions 150% of the expected former students 10 
Consumer- eligible to serve students | time of completion In the past but years after enrollment 
Facing Page receiving Financial Aid not currently: The 

under Title lV of the percentage of students 

Higher Education Act. in employment and 

(Includes post-college earning at least $28,000 

outcomes data only per year six years after 

on students receiving enrollment™ 

federal financial aid) 


4 $28,000 is based on ED’s estimate of the average earnings of individuals 25-34 years of age whose highest level of education is a high school diploma 
or its equivalent. 


° exit their program, and median earnings during the 
Th eC d OW N S | d es of 2™ quarter after exit. The Perkins V postsecondary 


consumer reports will also report employment 


curre nt consumer during the 2"? quarter after exit. In contrast, if a 


consumer looks at the consumer-facing pages 
re po rts of the College Scorecard they will see median 
Two major downsides of the current regime of earnings 10 years after enrollment. 
consumer reports are: 1) Most consumers and Of the three federal consumer report systems, 
other stakeholders do not see information on only the ETP consumer reports offer information 
earnings shortly after program completion; and on earnings shortly after program completion. It’s 
2) This regime creates an uneven playing field also the consumer report website that individuals 
that disadvantages CTE and training programs. are least likely to see. It’s most likely to be viewed 
If nothing is done to address this, the later_ by WIOA Title | participants who are interested in 
disadvantage will become more problematic when —_ exploring options for training. WIOA Title | funds 
the College Scorecard includes program-level training for only slightly more than 100,000 
information. individuals per year. 
If a consumer looks at an ETP consumer report The differences in the reports’ measures also 
they will see employment rates during the 2"° create an uneven playing field. Individuals who 
quarter and during the 4" quarter after students view the College Scorecard will see much higher 
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earnings because it shows earnings 10 years after 
enrollment. The longer the follow-up period for 
measuring earnings, the higher earnings tend to 
be. Earnings tend to increase as individuals have 
a longer history of employment. Furthermore, 

a period of 10 years after enrollment allows 

for additional education to occur unless the 
methodology excludes those who have attained 
additional levels of education. The College 
Scorecard makes no such exclusion. So, for 
example, the earnings for a B.A. in history will 
include those who continued their education, 
completed law school, and practiced law for a few 
years, not just those whose history B.A. is their 
highest level of attainment. The College Scorecard 
confounds the earnings outcomes for those in 
the labor market with a history B.A. with those 
whose highest level is a J.D. This type of problem is 
minimal for the shorter-term WIOA ETP measures 
since there is insufficient time for students to 
have attained additional degrees or longer-term 
certificates. 


The difference the follow-up 
period makes 

To illustrate the quantitative difference 

the follow-up period makes, we can look 

to Washington state which has data at the 
program level and includes many baccalaureate 
programs on its ETP consumer report website-- 
CareerBridge. For example, CareerBridge shows 
the median annual earnings for students with 

a history B.A. from the Eastern Washington 
University is $24,651 (four quarters after 
graduation). The state’s Education Research 
and Data Center’s (ERDC) Earnings for 
Graduates shows that students from the same 
program have median earnings of $38,300 
(four years after graduation). In another 
example, Washington’s CareerBridge shows 
that students with a psychology B.S. from 

the University of Washington have median 
annual earnings of $30,563 (four quarters 
after graduation), while the ERDC shows 

that students from the same program have 
median earnings of $50,700 (six years after 
graduation). 


Because of the different mix of programs reported 
by ETP and CTE consumer reports and the College 
Scorecard and their different follow-up periods, one 
could easily misunderstand the value of CTE and 
other training programs relative to baccalaureate 
and Associate of Arts Degree programs. 


Improving consumer 
reports 


A simple way of satisfying today’s students who 
want to know their likely earnings shortly after 
program completion, and making the playing 

field fair, is to include a shorter-term measure of 
earnings in the College Scorecard’s consumer- 
facing pages. This is the standard practice in higher 
education consumer reports established by states. 
A review of nine states’ higher education consumer 
report systems that show results at the program 
level, reveals that eight of the nine states include 
earnings approximately one year after completion 
at the front of the report. 


Including earnings approximately one year after 
completion or less, enables consumers to know 
how much they are likely to earn soon after 
completing a program, which is of interest to 
students in occupational programs, and of interest 
to anyone who thinks they might enter the labor 
market soon after they graduate. 


The College Scorecard relies on Treasury and So- 
cial Security Administration data for employment 
and earnings information. These sources can pro- 
vide annual, but not quarterly data.*° Using these 
data sources, the follow-up period that would be 
most similar to the shorter follow-up periods in the 
ETP and Perkins V consumer reports would be the 
calendar year following graduation. 


CTA would address this issue. The consumer 
report system established under CTA would report 
median and mean earnings immediately after 
leaving postsecondary education and at time 
intervals appropriate to the credential sought 

or earned.*© CTA also directs the Commissioner 

of Education Statistics to “minimize duplicate 
reporting across or within Federal agencies or 
departments, including reporting requirements 


5 Quarterly earnings data are available from states’ unemployment insurance wage records. WIOA requires wage records to be the primary source of 


data on performance measures of employment and earnings. 
6 College Transparency Act, Sec.2(3)(B)(iii)(I) and (II). 
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applicable to postsecondary institutions under the 
Workforce Innovation and Opportunity Act.””” This 
suggests that the “immediate” follow-up period 
could coincide with WIOA’s requirements. The 


exact metrics and methodology for CTA would be 
determined during implementation. 


Additional changes to the College Scorecard 

would also be helpful and could be included in the 
CTA reports as they are developed. As mentioned 
above, the College Scorecard mixes the earnings 

of those students who do and do not obtain 
additional education, so someone looking at the 
earnings does not know the distinct earnings for 
either group. Instead, the College Scorecard should 
report the earnings of those graduates who do not 
continue in additional education.”® 


Many individuals, of course, plan for a program to 
be a stepping stone to additional education. For 
example, this is the case for students enrolling in 
Associate of Arts Degree programs, or students 
enrolling in a four-year liberal arts program with 


2” College Transparency Act, Sec.2(3)(c)(i)(II)(bb). 
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the intention of going to graduate or professional 
school. Consumer reports should include the 
percentage of graduates who enroll in additional 
postsecondary education. Perkins V requires this 
for CTE consumer reports as would CTA, but the 
WIOA ETP reports and the College Scorecard 
consumer page do not include this. 


eo 


The Department of Education's 
College Scorecard with consumer- 


facing pages reporting earnings only 


during the tenth year after enrollment, 


is an outlier among postsecondary 
consumer reports. 


8 This is, for example, what Michigan’s Pathfinder consumer report system does. 
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Recommendations 


The recommendations below would provide 
consumers and others with information on 
postsecondary programs’ immediate labor 
market outcomes and level the playing field of 
consumer reports. ED can take these three steps 
by administrative action; no statutory changes are 


required.9 : 
The consumer-facing pages of 
™ The consumer-facing pages of the College the College Scorecard should 
Scorecard should include the median earnings Taveiece(sWualeMmaatsel rela) earnings of 
of graduates during the calendar year after graduates during the calendar 
graduation (or the second quarter after gradu- year after graduation (or the 
ation should quarterly data be available). second quarter after graduation 


should quarterly data be 
FW Fle) (9B 


™ Reported earnings should exclude gradu- 
ates who enroll in additional postsecondary 
education. 


™@ The Scorecard should include the percentage 
of graduates who enroll in additional postsec- 
ondary education. 


If CTA is enacted, the new consumer report system 
should include these recommended features. If 
Perkins or WIOA are opened for amendment, Congress 
should consider making aligned changes in these Acts’ 


reporting requirements. Reported earnings should 
(=y(od (Uo (=Wed e-lelUr-1e-S-M iV aleM-lal ce) 
Tame\e(elhd(oyat-1m elek-ia--terelatel-] avg 
education. 


The Scorecard should include 
the percentage of graduates who 
Valco] Miab-Ve(elietelat-1m ele-ie-t-(evelalel-]as 
education. 


22 NCES would need to match student data with National Student Clearinghouse data on postsecondary enrollments to identify graduates that enroll in 
additional postsecondary education. 
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postsecondary consumer reports 


WIOA eligible training providers” 
Total number of individuals exiting from the program of study (or the equivalent); 


The total number of participants who received training services funded by the adult and the dislocated 
worker programs disaggregated by the type of entity that provided the training, during the most recent 
program year and the 3 preceding program years; 


Total number of participants who exited from training services, disaggregated by the type of entity that 
provided the training, during the most recent program year and the 3 preceding program years; 


Average cost per participant for the participants who received training services, disaggregated by the 
type of entity that provided the training, during the most recent program year and the 3 preceding 
program years; and 


Number of individuals with barriers to employment served by the adult program and the dislocated 
worker program, disaggregated by each subpopulation of such individuals, and by race, ethnicity, sex, and 
age. 


Perkins “local recipient” report 
The percentage of CTE concentrators in career and technical education programs and programs of study 
that lead to non-traditional fields. 


Disaggregate data for each of the indicators of performance for the subgroups of students described in 
the Elementary and Secondary Education Act of 1965 and that are served under [Perkins], and quantify 
any disparities or gaps in performance, as described in paragraph 3(C)(ii)(II), between any such category 
of students and the performance of all CTE concentrators served by the eligible recipient. 


Disaggregate data, to the extent such data is available, for individuals in military service or a service 
program that receives assistance under title | of the National and Community Service Act of 1990 (42 
U.S.C. 12511 et seq.) or volunteers as described in section 5(a) of the Peace Corps Act (22 U.S.C. 2504(a)) 


College Scorecard Consumer Page 
Average annual net cost after financial aid 


By clicking on the consumer-facing page: 
Percent of students receiving a federal student loan; 
WW/Cete [reTalnteXe(=1e-] mel=)o)mr-)in=\aneae-(el6r-1u(0) an 
hs [gareie-\omagt=velt-lamanrealdalnm(ey-laMa-)er-\ anal ale 
Percentage of students who return after first year; 
Student body race and ethnicity 
Percent of students receiving Pell grants 


aM al=m @xe) |(=tex-McYoro)c=Yor-1 ae at-Km gale (ela manle)a-mel-it-Wr-\c-1] f-]e)(=mce) melon) (ex-leM oL6] mlm atom ofey-1X-\o Nola ke) al lial.c-1em Kom nal=) 
consumer-facing page. 


5° WIOA Sections 116 and 122. 
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